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Abstract 



We study the data transformation capabilities associated with schemas that are presented by labeled 
directed graphs and path equivalence constraints. Unlike most approaches which treat graph-based schemas 
as abbreviations for relational schemas, we treat graph-based schemas as categories. A morphism M between 
schemas S and T, which can be specified graphically as a mapping between graphs, induces three adjoint 
data migration functors, Ea/ : S — > T, 11^/ : 5 — > T, and Aa/ : T — >■ 5*. We present an algebraic query 
language FQL based on these functors, give decidability and complexity results, and present relational query 
generation algorithms. We prove that FQL is a schema transformation framework in the sense of Alagic and 
Bernstein, and is thereby suitable for common model management tasks like data warehouse construction 
and schema evolution. An understanding of category theory is not required for understanding the key ideas 
of this paper. 

1 Introduction 

In this paper we develop a model theory and operator algebra for database instances satisfying path equality 
constraints using the tools of category theory. In doing so we pick up a line of work initiated by Melnik. In 
his thesis [14], he describes three ways for achieving an "executable model theory" suitable for large-scale and 
generic information-integration efforts [15]; we quote these here: 

1. One way is to consider models and mappings as syntactic objects represented in a common meta-model, 
for example, as graphs. This approach has been pursued in almost all prior work on generic model 
management. In essence, the operators [model morphisms] are specified by means of graph transformations. 
As long as the graph transformations do not exploit any knowledge of what the graphs actually represent, 
the operators can be considered truly generic. Unfortunately, there are very few useful operations that 
can be defined in such an agnostic fashion. Largely, they are limited to Subgraph, Copy, and the set 
operations on graphs. 

2. A second way to achieve generic applicability is by using state-based semantics. In this approach, the prop- 
erties of the operators [model morphisms] are characterized in terms of instances of models and mappings 
that are taken as input and produced as output. Under the assumption that models possess well-defined 
sets of instances, all key operators can be characterized in a truly generic fashion. Such characterization 
is applicable to very complex kinds of models and mappings that are used in real applications, including 
XML Schemas, XQuery, and SQL. Although state-based characterization does not provide a detailed im- 
plementation blueprint, it is sufficiently specific so that the effect of the operators can be worked out for 
concrete languages. 

3. A third way for addressing generic applicability is an axiomatic one, e.g., using a category-theoretic 
approach. The idea of the approach is to define the operators [model morphisms] using axioms that are 
expressed in terms of the operators to be defined. Associativity of compose or commutativity of merge are 
examples of such axioms. This approach seems to be the most challenging, both in terms of determining 
a useful set of axioms and implementing the operators in such a way that the axioms hold when the 
operators are applied to concrete languages. 

Whereas Melnik proceeds to develop approach 2, and to implement a prototype called Rondo [16], in this 
paper we develop approach 3. In particular, we are concerned with the basic operations of the functorial data 
model [19]. 
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1.1 Background and Motivation 

In the data model of [19], database schemas are finitely presented categories [4]: directed labeled graphs with 
path-equivalences. Database instances are functors from categories/schemas to the category of sets. By targeting 
the category of sets, database instances can be stored as relational tables. Database morphisms are natural 
transformations (morphisms of functors) from database instances to database instances. The database instances 
and morphisms on a schema S constitute a category, denoted 5'-Inst. A morphism M between schemas S and 
T, which can be specified graphically as a correspondence between graphs, induces three adjoint data migration 
functors, T,m' S-Inst — >■ T-Inst, Hm ■ S'-Inst — T-Inst, and Am: T-Inst S'-Inst. At a high-level, this 
functorial data model provides an alternative category-theoretic foundation from which to study problems in 
information management. The mathematical foundations of this data model are developed in [19] using the 
language of category theory, but few specific connections are made to relational database theory. 

Our motivation for this work is that although labeled graphs with path equivalences are a very common 
notation for schemas [(>], there has been very little work to treat graphs as schemas directly. Instead, most schema 
transformation frameworks treat graphs as abbreviations for relational schemas, and treat path-equivalences as 
abbreviations for relational constraints. For example, in Clio [10], users draw lines connecting related elements 
between two schemas-as-graphs and Clio generates a relational query that implements the user's intended data 
transformation. Behind the scenes, the user's correspondence is translated into a formula in the relational 
language of second-order tuple generating dependencies, from which a query is generated [7]. As another 
example, in the Rondo system [IG], users are presented with an ad-hoc collection of operators over schema- 
as-graphs that they then script together to implement a data transformation. Although graphs and path- 
equivalences are used as inputs for both Clio and Rondo, both Clio and Rondo immediately translate from 
graphs and path-equivalences into a relational representation before proceeding further. 

1.2 Contributions and Outline 

In this paper, we 

1. Review the functorial data model of [19]. (Section 2) 

2. Define an algebraic query language FQL where every query denotes a data migration functor [19]. Every 
query in FQL can be written as three graph correspondences roughly corresponding to projection, join, 
and disjoint union. We show that FQL is closed under composition, and give decidability and complexity 
results. (Section 3) 

3. Provide a translation of FQL into the relational language of projection, selection, product, disjoint union, 
and key generation, where key generation constructs TV -I- 1-ary tables from A^-ary tables by adding a key 
or ID to each tuple. (Section 3) 

4. Prove that FQL is schema transformation framework in the sense of Alagic and Bernstein [2], thereby 
possessing other good formal properties that make it suitable for common model management tasks like 
data warehouse construction and schema evolution. (Section 4) 

We have implemented FQL in a prototype graphical schema mapping and query generation tool in the spirit 
of Clio and Rondo, available at wisnesky.net/fql.html. We conclude the introduction by reviewing some basic 
concepts in relational database theory and category theory. An understanding of category theory, while required 
to understand the details of some proofs, is not required to understand the main ideas of this paper. 

1.3 Review of Basic Relational Database Theory 

In our formalization of the relational model we assume finite product and co-product structure on our universe 
(i.e., domain, set of constants). More formally, a value is either a constant, like "David" or "Ryan", an n-tuple 
of values, or a value tagged with the name of a table. More formally: 

value ::— constant \ {value, . . . , value) \ value^ 

Equality of values is defined recursively, taking into account the tags; i.e. for values to be equal, tags must be 
equal. We write () for value-level tupling, and () for relation- level tupling. Columns in a relation are referred 
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to by position. A relational schema is a finite set R = {Ri, . . . , i?„} of relation names Ri and their associated 
arities. Our query generation algorithms target the following language: 

E ::= Ri \ 7r„i,..._„^ E \ am=n2 E \ keygen E \ \ TT^' 

i i 

In the above, UnEn indicates an rt-ary product of relations. Binary composition o and natural join xi can be 
defined using the above primitives: 

El o E2 := 7ri^4cr2=3(£;i x E2) Ei x E2 := tti, 240-1=3(^^1 x E2) (1) 

Let £^ be a relation of arity n. Then the semantics of keygen E, a relation of arity n+ I, is to add to each row 
a key column by forming a value-level tuple of a relation-level tuple: 

keygen{{{ai,bi), (02,62)}) = {((ai, 61), ai, 61), ((02, 62), 02, 62)} 

We adopt this purely functional semantics for key generation for convenience; however, imperative algorithms 
based on e.g., fresh identifiers can also be used. Melnik [14] similarly adopts key generation facilities. Note that 
key generation is only used in the queries generated from 11 and that value-level tupling only occurs when keys 
(single values) are created from tuples, (wi, . . . , u„) 1— )• (wi, . . . , 

We will target disjoint union instead of non-disjoint union. Suppose given n tables and 2n tags (one string 
for each column), T: {1, . . . , 2n} — > Str, the symbol ]J indicates n-ary disjoint union of binary tables, param- 
eterized by these tags. For example, let i?i = {(ai, &i), (02, ^2)} and R2 = {(ci, C2)} be relations of arity 2, and 
let our tags be (s, t, u, v). Then the semantics of JJ'*^*'"^" i?2, a relation of arity 2, is: 

s.t,u.v 

H i?i,i?2 = {(a?,&*i),(4,6*2),(cr,c^)} 

We will only apply disjoint union to relations of arity 2, and our tags will always be table names. Note that 
disjoint union is only used in queries generated from E. 

1.4 Review of Basic Category Theory 

Category theory is an axiomatically specified algebra of abstract functions suitable for formalizing mathematics. 
In contrast to traditional set-theory, where structures are defined by what they are, in category-theory structures 
are defined by how they interact. Compared to set-theory, category theory is notable for its high-level of 
abstraction and focus on compositionality. Since its inception in 1945, category theory has been applied in many 
disciplines, including information management, where categorical methods inspired the design of functional 
query languages such as the nested relational algebra [20]. The adjacent fields of mathematical logic and 
programming language theory also employ categorical techniques [13]. 

A category C consists of a class of objects Ob(C) and a class of morphisms or arrows homiC) between 
objects. Each morphism m has a source object S and a target object T, which we write as m: 5* — >■ T or 

S T. Every object X has an identity morphism idx : X ^ X. Two morphisms f:B^C and g: A ^ B 
may be composed, written f o g: A ^ C . Composition is associative and id is its unit: 

foid=d idof = f f o{goh) = {f og)og 

A morphism / : X — > y is an isomorphism when there exists another morphism g: Y ^ X such that 

f o g ^ id g o f = id 

Two objects are isomorphic when there exists an an isomorphism between them. A functor _F : C — > D between 
two categories C and D is a mapping of objects of C to objects of D and morphisms of C to morphisms of D 
that preserves identities and composition: 

F{f -.X^Y): Fix) ^ F{Y) F{id) = id F{f o g) = Fif) o F{g) 

A natural transformation between two functors _F : C ^ D and G : C — >■ D is a family of morphisms ax '■ F{X) 
G{X) in D, one for each X G Ob(C), such that for every morphism f : X ^ Y in C 

ay o F{f) = G(/) o ax 
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The natural transformation a is an isomorphism when for every object X in C, the morphism ax is an 
isomorphism in D. Two functors F and G are called (naturally) isomorphic if there exists an isomorphism from 
F to G. 

Almost all of the category-theoretic content of this paper can be understood using the above concepts. The 
proofs, however, are much more sophisticated. Example categories include Set, the category of sets and total 
functions; Grp, the category of groups and group homomorphisms; and Cat, the category of categories and 
functors. An example functor is the power set functor V: Set — > Set, which maps each set to its power set 
and each function f : X Y to the map which sends U C X to its image f{U) C Y. Because our database 
instances on a schema S are functors S — > Set, it is appropriate to define our database morphisms to be natural 
transformations: a database morphism a^: Ii{R) l2{R) is a mapping of values in Ii{R) to values in l2{R)- 
Hence, our database morphisms are "multi-sorted" traditional relational homomorphisms; see remark 2.1.1 for 
details. 



2 Functorial Data Migration 

The functorial data model [19] uses directed labeled graphs and path equalities for signatures. A path p is 
defined inductively as: 

p ::= node \ p. edge 

A signature 5 is a finite presentation of a category. That is, a signature is a triple 5* := {N, E, C) where N is 
a finite set of nodes, _E is a finite set of labeled directed edges , and C a finite set of path equivalences. E.g, 



manager 




worksin Dept 

(2) 



secretary 



DName 



Emp.meinager. worksin = Emp. worksin 

Dept. secretary. worksin = Dept 

Here we see a signature S with five vertices and six arrows, and two path equivalence statements. This 
information generates a category [S"]]: the free category on the graph, modulo the path equivalence relation. 
The category |[5']| is the schema for S, and database instances over \SJ are functors [S*] — ?> Set. Every path 
p: X — >■ F in a signature S denotes a morphism [p]] : \XJ [[FJ in [S*]. Given two paths pi,p2 in a signature 
S, we say that they are equivalent, written pi = p2 if Ipil and [[P2I are the same morphism in [[S*]). Two 
signatures S and T are isomorphic, written 5* = T, if they denote isomorphic schema, i.e. if the categories they 
generate are isomorphic. 

2.1 Binary Representation of Instances 

Our database instances can be represented as relational database instances in several ways, but we will generally 
use the following functional binary representation, in which every table has two columns: 

• To each node N corresponds an "identity" or "entity" table named N, a reflexive table with tuples of the 
form (x,x). We can specify this using first-order logic: 

\/xy.N{x,y) ^ X = y. (3) 

• To each edge e: Ni N2 corresponds a "link" table e between identity tables Ni and A^2- The axioms 
below merely say that every edge e: Ni N2 designates a function A^i — > A^2 : 

Vxy. e{x, y) =^ Ni{x, x) \/xy. e{x, y) ^ N2{y, y) 

\lxyz. e(x, y) A e{x, z) ^ y = z Va;. Ni{x, x) 3y.e{x, y) 
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Here is an example instance on the above schema (2), 
tables and six link tables. 



written in binary form. As expected, it has five entity 



FName 


ID 


ID 


Alan 


Alan 


Camille 


Camille 


Andrcy 


Andrey 



LName 


ID 


ID 


Turing 


Turing 


Jordan 


Jordan 


Markov 


Markov 



Emp 


ID 


ID 


101 


101 


102 


102 


103 


103 



DName 


ID 


ID 


Math 


Math 


CS 


CS 



Dept 


ID 


ID 


qlO 


qlO 


x02 


x02 



first 


Emp 


FName 


101 


Alan 


102 


Camille 


103 


Andrey 



last 


Emp 


LName 


101 


Turing 


102 


Jordan 


103 


Markov 



name 


Dept 


DName 


x02 


Math 


qlO 


CS 



manager 


Emp 


Emp 


101 


103 


102 


102 


103 


103 



worksin 


Emp 


Dept 


101 


qlO 


102 


qlO 


103 


x02 



secretary 


Dept 


Emp 


x02 


102 


qlO 


101 



The binary representation has the advantage that path expressions correspond directly to compositions of binary 
tables. Corresponding to a path expression p is the following relational query [p] that "follows" or "de-references" 
p by composing tables, where binary relational composition o is defined as in (1): 

[node] node 

[path. edge] :— [path] o [edge] (5) 
Hence, a path equivalence corresponds to an equality of two relational queries, 

[pi = P2] := bi] = [P2] (6) 

for example, 

[Emp.manager. worksin = Enip.worksin] := Emp o Manager o worksin = Emp o worksin 

Throughout the paper we will use the notation [] to indicate relational counterparts to functorial expressions. 

Let S" be a signature, and let [S] be its image as a relational schema under the above translation. Add to 
[S] the functional axioms (3) (4) above. Then the following theorem states that every ^'-instance (functor from 
S to Set) can be represented as a set of tables conforming to [S], and every set of tables conforming to [S] 
represents an S'-instance (functor from S to Set). 

Theorem 1. Every S -instance is an [S]-instance, and every [S]-instance is an S -instance. 

Proof. Proved as Theorem A. 1.1. □ 

Remark 2.1.1. For us, a database morphism an'. Ii{R) ^2(-R) is a natural transformation: for each R, a 
mapping of values in Ii{R) to values in l2{R), respecting foreign keys. A traditional database morphism is a 
mapping of values in Ji to values in I2. In other words, our database morphisms are multi-sorted: we have one 
mapping for each node table, rather than one mapping across all tables. 

Remark 2.1.2. The relationally-minded reader may be concerned that we are mixing "meaningful" constants like 
Alcin and "meaningless" constants like x02. Most of the constructions in this paper characterize instances only 
up to isomorphism; hence, all constants should be considered "meaningless". However, the binary representation 
of instances provides a trivial way to associate a human-meaningful (or otherwise useful) annotation to each 
ID in a node table: store the annotation in the redundant second column. Part of the example above would 
become: 



Emp 


ID 


Title 


101 


Computer Scientist 


102 


Mathematician 


103 


Statistician 



first 


Emp 


FName 


101 


1 


102 


2 


103 


3 



FName 


ID 


Name 


1 


Alan 


2 


Camille 


3 


Andrey 



2.2 A^-ary representation of Instances 

An alternative to binary form is so-called categorical normal form [19]. In this representation, the first column 
of every table is an ID column. To convert a binary instance to this form, identity tables lose their redundant 
second column, and we keep only those that are "leaf" tables, i.e. have no outgoing arrows (Melnik calls such 
unary tables "selectors" [14]). Then, for each node N, all binary tables Ei: N Yi are naturally joined together 
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to become a single "fact" table with columns 1^. Converting from categorical normal form to binary form is 
a traditional join-decomposition [1]. In this paper we will use both the binary form and categorical normal 
form as needed. In particular, categorical normal form is better for presenting example instances (it is more 
succinct), but binary form is better for generating queries (it is more uniform). 



FName 



ID 



Alan 



Andrey 
Camillo 



LName 



ID 



Jordan 



Markov 
Turing 



DName 



ID 



Math 
"CS 



Emp 


ID 


first 


last 


mgr. 


worksin 


101 


Alan 


Turing 


103 


qlO 


102 


Camille 


Jordan 


102 


x02 


103 


Andrcy 


Markov 


103 


qlO 



Dept 


ID 


name 


sec. 


qlO 


CS 


101 


x02 


Math 


102 



2.3 Schema Mappings 

Let C and T) be signatures. A signature morphism _F : C — > I? is a mapping that takes vertices in C to vertices in 
T) and arrows in C to paths in I?; in so doing, it must respect arrow sources, arrow targets, and path equivalences. 
In other words, if pi = p2 is a, path equivalence in C, then F(pi) = F(p2) is a path equivalence in V. Each 
signature morphism F: C V determines a unique schema morphism [i^] : [C] — >■ [P]] in the obvious way. 
Two signature morphisms i^i : C — > I? and F2 : C — > I? are equivalent, written Fi = F2, if they denote isomorphic 
functors. Below is an example of a signature morphism. 



C 





Each of the four leaf vertices, SSN, First, Last, and Salary in C is mapped to the vertex in V of the same 
label. The two other vertices in C, namely Tl and T2, are mapped to vertex T in P. Since morphisms must 
respect arrow sources and targets, there is no additional choice about where the arrows in C are sent; for example 
F sends the arrow Tl — > First in C to the arrow T — > First in V. 



2.4 Data Migration Functors 

Each signature morphism F: C ^ V is associated with three data migration functors, Ap, Yip, and 11^, which 
migrate instance data on V to instance data on C and vice versa. 

Definition 1 (Data Migration Functors) . Let F : C ^ V be a functor. We will define a functor A p : V -Inst ^ 
C-Inst; that is, given a V-instance /: I? — > Set we will construct a C-instance Ap{I). This is obtained simply 
by composing the functors I and F to get 



Ap{I) -.^loF-.C^ Set 




Then, Yp : C-Inst — > I?-Inst is defined as the left adjoint to Ap, and Hp : C-Inst — >■ I?-Inst is defined as the 
right adjoint to Ap. 

An adjoint is similar to an inverse; see definition B.1.1 for the exact definition. We have the following 
summary of the three data migration functors: 
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Data migration functors induced by a translation F: C ^ V 


Name 


Symbol 


Type 


Idea of definition 


Relational partner 


Fullback 


Af 


Af: Inst C Inst 


Composition with F 


Project 


Right Pushforward 


Up 


Up: C-Inst P-Inst 


Right adjoint to Af 


Natural join, Keygen 


Left Pushforward 




Sf: C-Inst P-Inst 


Left adjoint to Af 


Disjoint union 



We now examine each of these operations in turn. 



2.5 Fullback - Ap : P-Inst C-Inst 

Even though our translation F points forwards (from C to V), our migration functor Af points "backwards" 
(from I?- instances to C-instances) . Consider the instance J, on schema V, defined by the table 



T 


ID 


SSN 


First 


Last 


Salary 


XF667 


115-234 


Bob 


Smith 


$250 


XF891 


122-988 


Sue 


Smith 


$300 


XF221 


198-877 


Alice 


Jones 


$100 



(7) 



Af(J) splits up the columns of table T according to the translation F. The four leaf tables will be unchanged, 
and the two fact tables will be: 



Tl 


ID 


SSN 


First 


Last 


XF667 


115-234 


Bob 


Smith 


XF891 


122-988 


Sue 


Smith 


XF221 


198-877 


Alice 


Jones 



T2 


ID 


First 


Last 


Salary 


XF221 


Alice 


Jones 


$100 


XF667 


Bob 


Smith 


$250 


XF891 


Sue 


Smith 


$300 



The fact that Tl and T2 are simply projections of T is a result of our choice of translation F. We will refer to 
the above instance as / := Af(J). 



2.6 Right Pushforward — Up: C— Inst — > X>— Inst 

Our migration functor Hp points in the same direction as F: it takes C-instances to P-instances. It turns out 
that ^f{I) - nFAF(J) = J; in other words ^f{I) is isomorphic to (7). Note that this will not be equal, but 
we can use the annotation as discussed in Remark 2.1.2 to display (7), as desired. In fact for this functor F, 
the data migration functor IIf serves as a left-inverse to Af, up to isomorphism. This is not true in general, 
but there is for arbitrary G: C ^ V a map J UgAqJ for any P-instance J. 

In general, 11 will take the join of tables, as parameterized by V. See [19, Example 2.2.3] for a more interesting 
example of the data migration functor 11, and see Construction B.2.14 for an implementation of 11 in terms of 
joins. 



2.7 Left Pushforward - Si.: C-Inst P-Inst 

Our migration functor Sf points in the same direction as F: it takes C-instances to I?-instances. We now 
describe the P-instance Sf(/), where I is the instance above. Sf(/) has four leaf tables Sf(^)(SSN), etc., and 
one fact table Sf(/)(T). Instead of being a join, as in the case of I1f{I) above, the fact table T in instance 
Sf(/) will be the disjoint union of Tl and T2. To deal with the fact that records in Tl do not have salary 
information and the records in T2 do not have SSN information (the two relations are not union-compatible in 
the sense of Codd), we use labeled nulls or skolem functions. Thus in instance Sf(/), table T looks like this: 



T 


ID 


SSN 


First 


Last 


Salary 


XF667'^i 


115 - 234'^ i 




Smith'-''^ 


XFeer.Salary^'^ 


XF891-'i 


122 - 988'-' i 


Sue'-'^ 


Smith'-' 1 


XFSm.Salary^'^ 


XF221-'i 


198 - 877'-' i 




Jones^ 1 


XF221.Salary^'^ 


XF221'^'2 


XF221.SSN'-^'-2 




Jones^^ 


$100'^'2 


XF667'^'2 


XFmr.ssN'^'^ 




Smith'-'''^ 


$150'^'2 


XF891-''2 


XF891.SSN'-^'^ 


Sue'-'^ 


Smith'-''^ 


$300'-' 2 



The labeled nulls (such as Tl-001. Salary) can be equated with actual values later. They can also be equated 
with each other; for example we may know that Tl-001. Salary=T2-002. Salary, without knowing the value of 
these salaries. In the next section, where we define a query language based on these data migration functors. 
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we place sufficient restrictions on F to ensure that labeled nulls are never needed; i.e., we will only take disjoint 
unions of relations with the same arities - see definition 4. Note that there is a database morphism Tip J — > /; 
this is always and canonically the case. 

Remark 2.7.1. Category theory's focus on isomorphism instead of equality can sometimes be counterintuitive, 
at least from a relational perspective. For example, although we have that Aid(/) = / and o = I^foGi 
this is not true for 11 and E; instead we have that Tlid{I) = I and Yip o YIq = Ili^oGi a-nd that Y,id{I) = I and 
Yip o Eg = YpoG- For this reason, we will be careful to distinguish isomorphism from equality. 

In practice, it can sometimes be cumbersome for users to work with instances up to isomorphism. Intuitively, 
one might start with a database instance / containing the constant "George", only to find that which 
isomorphic to /, contains the constant "128XYG" instead. This behavior can be problematic when "constants 
matter" (e.g., someone's name), but beneficial when "constants shouldn't matter" (e.g. someone's patient ID 
number). Fortunately, it is trivial to annotate each key with a "human meaningful" value: instead of associating 
each node in a schema to a binary reflexive table, we use the second redundant column to store the human 
meaningful value; see remark 2.1.2 for details. The query generation algorithms we present in the next section 
can be easily adapted to preserve these annotations. 



3 FQL: A Query Language for Functorial Data Migration 

The goal of this section is to define and study an algebraic query language where every query denotes a compo- 
sition of data migration functors. Our syntax for queries is as follows. 

Definition 2 (Data Migration Query). A data migration query Q from S to T, denoted Q: S T is a triple 
of signature morphisms {F,G,H): 

S^S' ^S" 

Semantically, the query Q : S T corresponds to a functor [Q]] : [S'J-Inst — >■ [[TJ-Inst given as follows: 

IQl := Ei^fjniGiAjFj: [^J-Inst ^ |[T]]-Inst 

By choosing two of F, G, and H to be the identity mapping, we can recover A, S, and 11, up to isomorphism. 
However, grouping A, S, and 11 together like this formalizes a query as a disjoint union of a join of a projection. 
When H is the identity morphism, we say that a query is conjunctive. 

Definition 3 (Morphism of Queries). A morphism of data migration queries h : Qi ^ Q2 is a natural transfor- 
mation [QiJ — > IQ2l- Two queries Qi and Q2 are isomorphic, written Qi - Q2, when there exists morphisms 
Qi Q2 and Q2 ^ Qi- 



3.1 Restricted Queries 

Data migration queries do not seem to be closed under composition.^ However, we can place certain restrictions 
on the signature morphisms that make up a query. To understand the nature of the restriction, we will need 
the following definition. 

Definition 4 (Discrete op-fibration). A functor F: C ^ V is called a discrete op-fibration if for every object 
c € Ob(C) and every arrow g: d ^ d' in V with F{c) — d, there exists a unique arrow g: c ^ c' in C such that 

F{-g) = g. 

The following example, which maps as to A, bs to B, cs to C, gs to G, and hs to H, is a discrete op-fibration: 



bl 91 ai hi d 



b-2 92 a2 ^2 C2 




^The issue is a difficulty in phrasing an appropriate analogue of the distributive law, Proposition B.3.17 when u is not a discrete 
op-fibration. There will be no straightforward way to do this, as shown by Example B.3.18. 
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Let Q := {F, G, H) be a data migration query, so that [QJ = T,h^g^f- Then when F (or H or both F and 
H) is a discrete op-fibration, we say that Q is A-restricted (or Ti-restricted or (A, S)-resirzcted). The following 
theorem says that S-restricted queries are closed under composition. Theorem B.4.10 says that (A, S)-restricted 
queries are most suitable for syntactic analysis. 

Theorem 2 (Closure under composition). Given any H-restricted (respectively (A, -restricted) data migration 
queries f : S X and g: X T, there exists a Yi-restricted ( respectively (A, S) -restricted) data migration query 
g ■ f '■ S T such that 

l9-fl = l9l o I/l. 

Sketch of proof. Following the proof in [9, Proposition 1.1.2], it suffices to show the Beck-Chevalley conditions 
hold for A and its adjoints, and that dependent sums distribute over dependent products when the former are 
indexed by discrete op-fibrations. The proof and algorithm are given in the appendix as Theorem B.3.19 and 
Corollary B.3.20. □ 

Because the fiber product of two cyclic finitely-presented categories may not be finitely presented, it is unclear 
if composition queries are computable when schemas are cyclic. 

3.2 Query Generation 

In this subsection, we prove that every E-restricted data migration query can be implemented using projection, 
selection, disjoint union, product, and key generation. Let F: 5' — >■ T be a signature morphism. 

Theorem 3. We can compute a set of queries [F]^: T — > S* such that for every T-instance I e T-Inst, we 
have Af{I) = [F]a{I)- 

Proof. See Construction B.2.1. □ 

Theorem 4. Suppose F is a discrete op-fibration. Then we can compute a set of queries [F]^: S such 
that for every S-instance I € S'-Inst, we have T,p(I) = [F]y:{I). 

Proof. See Construction B.2.10. □ 

Theorem 5. Suppose ^SJ and [T]] are finite. Then we can compute a set of queries [Fju- S" — > T such that 
for every S-instance I G S'-Inst, we have IIf{I) = [-F]n(-^)- 

Proof. See Construction B.2.14. □ 

When IS*] or [T] are infinite, query generation for 11 is problematic because on finite inputs 11 may create 
uncountably infinite result instances. See Example B.2.15. If we are willing to take on countably infinite 
relational queries, we have 11 in general, as follows: 

Theorem 6. There exists a set of countably infinite queries [Fju : 5 — > T such that for every S-instance 
I G S'-Inst, we have 0^.(7) = [F]n(/). 

Proof. See Construction B.2.14. □ 

Alternatively, it is possible to target a category besides Set. For example, it is possible to store our data not as 
relations, but as programs in the Turing-complete language PCF [19] [17], which is complete and cocomplete as 
a category. In such a case, our data migration functors A, E, and 11 would denote continuous functions between 
partial orders and be implemented by a finite but recursive program. 



3.3 Decidability and Complexity Results 

Our notion of schema is precisely that of finitely presented category. In the presence of loops, such cate- 
gories/schemas may have infinitely many morphisms. For purposes of analyzing decidability and computational 
complexity we must distinguish two undecidable problems: 

• The word problem for categories asks if two paths in a finitely presented category are equal. The "comple- 
tion without failure" extension [3] of the Knuth-Bendix algorithm provides a semi-decision procedure for 
this r.e. problem [8]. The Knuth-Bendix algorithm first attempts to construct a re-writing system based 
on the path equalities; if it succeeds, it yields a linear time decision procedure for the word problem [11]. 
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• The equivalence problem for categories asks if two finitely presented categories are equivalent. Equivalence 
of categories is a weaker notion than isomorphism of categories. An equivalence of categories C and V 
consists of two functors F : C ^ T) and G : T) ^ C, and two natural isomorphisms F o G I-d and 
7c — >■ G o F, where Iq :€—>■€ and I-p : V ^ V are the identity functors on C and V. We have been 
unable to determine the exact computational complexity of this problem. 

The word and equivalence problems for finite categories, such as generated by acyclic schema, are decidable. 
We use the word problem to check if a mapping between signatures preserves path equivalences, and we use the 
equivalence problem to determine if two restricted data migration queries are are isomorphic. 

Theorem 7 (Complexity of Query Isomorphism). Let Qi and Q2 be (A,Yi)-restricted data migration queries. 
Then the complexity of finding an isomorphism Qi O Q2 is at most the complexity of testing equivalence of 
categories. 

Proof. This is proved as Corollary B.4.11. □ 

We provide a condition for determining if a signature morphism is a discrete op-fibration in Proposition 
B.3.11. This condition uses both the word and equivalence problems, but we cannot determine its exact 
complexity without a characterization of the complexity of the equivalence problem for categories. 

4 Functorial Data Migration as Schema Transformation 

In this section we prove that the functorial data model is a schema transformation framework in the sense of 
Alagic, Bernstein and Melnik [2] [14]. At a high level, when a data model possesses the structure of a schema 
transformation framework, it is compatible with "generic model management" or "abstract model theory" and 
hence is expressive enough to capture many common information integration scenarios. Typically, such systems 
are used to partially automate tasks in large-scale information management efforts, such as data warehouse 
construction, deep analysis of semi-structured data, schema evolution, and query generation from high-level 
schema correspondences [5]. 

It is easy to describe basic functorial data migration using Alagic and Bernstein's categorical model the- 
ory [2]. However, there is a fundamental difference in terminology. In this paper, we use "signature" for a 
finitely presented category, and "schema" for the potentially infinite category it denotes. Alagic and Bernstein 
leave "signature" undefined, and use "schema" for a pair of a signature and a set of sentences in some logi- 
cal formalism. Hence, every schema in our sense can be represented as a potentially infinite schema in their 
sense (its presentation), and every signature in our sense can be represented as a finite schema in their sense 
(its finite presentation). We consider path-equivalences to be part of our signatures, so our instances obey 
path-equivalences by definition; in their system, path-equivalences are not part of signatures and a separate sat- 
isfaction relation \=s characterizes the path-equivalences that hold in an S'-instance. In the following definition, 
we use "signature" and "schema" in their sense. Let L be a logical formalism such as first-order logic. Then 

Definition 5. A schema transformation framework is a tuple {Sig, Sigo, Sen, Db, |=) such that 

1. Sig is a category, representing signatures together with their morphisms, and Sig has an initial object 
Si go. 

2. Sen : Sig — >■ Set is a functor such that Sen(Sig) is the set of all L-sentences over the signature Sig. 

3. Db: Sig — > Cat°^ is a functor sending each signature Sig to the category Db{Sig) of Sig -instances and 
their morphisms. 

4-. For each signature Sig, the satisfaction relation \=sig C \Db{Sig) \ x Sen(Sig) is such that for each schema 
signature morphism (p: Sig a — > Sign, the following integrity requirement holds for each Sign database ds 
and each sentence e € Sen(SigA) '■ 

db hsige Sen{(l)){e) iff Db{(j)){db) \^sigA e (8) 

The integrity requirement (8) puts a strong semantic restriction on the permissible schema morphisms: it 
only allows ones that preserve the validity of constraints — exactly like schema morphisms in the functorial 
data model. 
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Theorem 8. The functorial data model is a schema transformation framework. 
Proof. We use the following definitions: 

1. Sig is a the category of finitely presented freely generated categories, and Sigo is the empty category. 

2. Sen(Sig) is the set of all equations over the signature Sig. 

3. Db: Sig Ca.t°P is A (see Section 2.5). 

4. / 1= pi = p2 is defined in the intuitive way. 

The result follows from Lemmas 1, 2, 3, and 4 below. □ 
Lemma 1 (Requirement 1). Signatures form a category with an initial object. 

Proof. Proved as Lemma A.L2. □ 

Lemma 2 (Requirement 2). We have a functor Sen: Sig — > Set such that Sen(Sig) is the set of all equations 
between generators over the signature Sig. 

Proof. Proved as Lemma A.L3. □ 

Lemma 3 (Requirement 3). The assignment Sig i— > Db(Sig) is functorial. That is, for each morphism 
of signatures F: Sig — > Sig' there is an induced morphism Db(Sig) — >■ Db(Sig') in Ca.t°^ , i.e. a functor 
Db(F) : Db(Sig') Db(Sig). We write Db: Sig — > Cat°'', or as is slightly easier to parse, 

Db: Sig°P ^ Cat. 

Proof. Proved as Lemma A.L4. □ 

Lemma 4 (Requirement 4). For each schema signature morphism (j): Sig a — > Sigs, the following statement 
holds for each Sigs instance ds and each path equation e G Sen{SigA)'- 

ds hsigB Sen{(p){e) iff Db{(p){dB) hsig^ e 

Proof. In categorical terms, the above says that given a functor (f>: A ^ B and a functor ds - B ^ Set, the 
composition I o (p: A — > Set is a functor. See A.L5 for the full proof. □ 

Moreover, our schemas have a further useful schema-join property property: 

Theorem 9. There exists a schema join of any two schemas over any subschema (in particular over the initial 
schema). 

Proof. Given a diagram of schemas B ^ A ^ C, the schema join (defined as the colimit in Cat) always 

exists. □ 

In this section we have only proved that the functorial data model is a schema transformation framework. 
However, it follows that the A fragment of FQL is a schema transformation framework. It may be the case that 
larger fragments of FQL are schema transformation frameworks; see the remarks around Definition B.4.5. 



5 Conclusion 

In this paper we have defined an "executable model theory" suitable for large-scale information integration tasks 
using the tools of category theory, and have implemented our theory in a prototype schema mapping and query 
generation tool. Our work connects traditional relational database theory to category theory in a new way, and 
leaves many interesting avenues unexplored. In particular, the natural sense in which our theory connects with 
existing work on generic model management suggests that our categorical approach might be applicable to a 
wide variety of problems in information management. 
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A Basic proofs 



In this section, wc fill in those leftover proofs from the main text that do not require much category theory. 

Theorem A. 1.1. Let S be a signature, and let [S] be its image as a relational schema as in Section 2.1. Then 
every S -instance is an [S]-instance, and every [S]-instance is an S -instance. 

Proof. A functor [S] — >■ Set assigns to each object a set and to each arrow a function, such that composition is 
preserved. There is a one-to-one correspondence between sets N and identity tables as in (3) and a one-to-one 
correspondence between functions iVi — >■ N2 and link tables as in (4). Finally, these sets and functions preserve 
composition if and only if they satisfy the path equivalence axiom as in (6). □ 

The following lemmas prove the four requirements for a schema transformation framework, as in Definition 
5 and discussed in Section 4. 

Lemma A. 1.2 (Requirement 1). Signatures form a category with an initial object. 

Proof. The empty category is initial in the category of finitely presented freely generated categories. This 
corresponds to an ambient universal domain. Multiple atomic types like strings, integers, and booleans can be 
supported through a simple slice category construction. □ 

Lemma A. 1.3 (Requirement 2). We have a functor Sen: Sig — > Set such that Sen(Sig) is the set of all 
equations over the signature Sig. 

Proof. For each Sig G Sig, we take the set of all finite path expressions X.pi p„, close them under compo- 
sition, and then enumerate the set of equations between such terms. So the object part of Sen(Sig) is the set 
of all path equations. For each 0: Sigi — >■ Sig2, we associate a function Sig{(j)) : Set — >■ Set = Xeqs.(j){eqs) that 
"applies" (j) in the obvious manner, yielding a transformed set of equations. □ 

Lemma A. 1.4 (Requirement 3). The assignment Sig 1— > Db{Sig) is functorial. That is, for each morphism 
of signatures F: Sig — >■ Sig' there is an induced morphism Db{Sig) — ^ Db{Sig') in Cat°P, i.e. a functor 
Db{F) : Db{Sig') — >■ Db{Sig). We write Db: Sig — >■ Cat°P, or as is slightly easier to parse, 

Db: Sig°P ^ Cat. 

Proof. Let F: Sig — > Sig' be a functor between free categories. Given an object in Db{Sig'), i.e. a functor 
d: Sig' — >■ Set define Db{F) = do F: Sig — > Set. Given a morphism in Db{Sig'), i.e. a natural transformation, 
compose it with the identity natural transformation on F to get a morphism in Db{Sig). One checks easily that 
with these definitions, Db is indeed a functor. □ 

Lemma A. 1.5 (Requirement 4). For each schema signature morphism <f>: SigA — >■ Sigs, the following state- 
ment holds for each Sigs instance dg and each path equation e G Sen(SigA): 

ds \=StgB Sen{(t)){e) iff Db{(t)){dB) \^sigA e 

Proof. We are given an instance ds on B. By definition, for any table X in A, we have Db{4)){dB){X) = 
Db{(t){X)) and for any path p: X ^ Y in A and row x G Db{(j){X)), we have 

Db{cl,){dB){p){x) ^ dBiHPm^) (9) 

Suppose e equates paths p,q: X ^ Y for tables X,Y in A. By definition dB \=SigB Sen{(p){e) iff for all 
X G dB{4>{X)) we have dB{<j>{p)){x) = dB{<j>{q)){x). By (9), this is the case iff for all x G Db{<l)){dB) we have 
Db{(t)){dB){p){x) = Db{(f)){dB){q){x), which is by definition what is needed, Db{(f){dB) \=SigA ^■ 

□ 

B Categorical constructions for data migration 

Here we give many standard definitions from category theory and a few less-than-standard (or original) results. 
The proofs often assume more knowledge than the definitions do. Let us also note our use of the term essential, 
which basically means "up to isomorphism". Thus an object X having a certain property is essentially unique 
if every other object having that property is isomorphic to X; an object X is in the essential image of some 
functor if it is isomorphic to an object in the image of that functor; etc. 
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B.l Basic constructions 



Definition B.1.1 (Adjoints). An adjunction between categories C and D consists of 

• A functor F : D ^ C called the left adjoint 

• A functor G : C D called the right adjoint 

• A natural isomorphism (/) : homc{F^, — ) — > homjj^—, G~) 

• A natural transformation e : FG — > Ic called the counit 

• A natural transformation 77 : — >■ GF called the unit 

Definition B.l. 2 (Fiber products of sets). Suppose given the diagram of sets and functions as to the left in 
(10). 



A 



B 



C 




(10) 



/ / 

Its fiber product is the commutative diagram to the right in (10), where we define 

AxcB:^{{a,cM \ f{a) = c = g{b)} 

and /', g' , and h = g o f = f o g' are the obvious projections (e.g. /'(a, c, b) = b). We sometimes refer to just 
{A Xc B, f ,g' , h) or even to A S as the fiber product. 

Example B.l. 3 (Chain signatures). Let n € N be a natural number. The chain signature on n arrows, denoted 
~rt, is the graph 



with no path equivalences. One can check that it has ("J^)-many paths. 

The chain signature is the terminal object in the category of signatures. _^ 

A signature morphism c : — >■ C can be identified with an object in G. A signature morphism / : 1 — > C 



can be identified with a morphism in G. It is determined up to path equivalence by a pair (n,p) where n € N 
is a natural number and p: ~rt — > C is a morphism of graphs. 

Definition B.1.4 (Fiber product of signatures). Suppose given the diagram of signatures and signatures 
mappings as to the left in (11). 



A- 



B 



G 



Axr; B 




(11) 



/ / 

On objects and arrows we have the following diagrams of sets: 

Oh{B) 



Arr(B) 



Ob(^) 



Ob(/) 



Ob(g) 

Ob(C) 



Arr(g) 



(12) 



Arr(A) 



Arr(/) 



Mor(C) 



We define the fiber product of the left-hand diagram in (11) to be the right-hand commutative diagram in 
(11), where we the objects and arrows of A Xc B are given by the fiber product of the left and right diagram 
of sets in (12), respectively. A pair of paths in A Xc B are equivalent if they map to equivalent paths in both 
A and B (and therefore in C). Note that we have given A Xc B as a. signature, i.e. we have provided a finite 
presentation of it. 



^The chain signature n is often denoted [n] in the literature, but we did not want to further overload the bracket [-] notation 
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Example B.1.5 (Pre-image of an object or morphism). Let F: C — > D be a functor. Given an object d e Oh{D) 
(respectively a morphism d G Mor(Z?)), we can regard it as a functor d: — > _D (respectively a functor 
d: 1 ^ D), as in Example B.1.3. The pre-image of d is the subcategory of C that maps entirely into d. This 

F d — ^ F d — ^ 

is the fiber product of the diagram C — > D <— (respectively the diagram C — > D <— 1 ). 

A signature is assumed to have finitely many arrows, but because it may have loops, C may have infinitely 
many non-equivalent paths. We make the following definitions to deal with this. 

Definition B.1.6. A signature C is called finite if there is a finite set S of paths such that every path in C is 
equivalent to some s G S. An equivalent condition is that the schema HCJ has finitely many morphisms. (Note 
that if C is finite then C has finitely many objects too, since each object has an associated trivial path.) 

Recall (from the top of Section 2) that every signature is assumed to have a finite set of objects and arrows. 
The above notion is a semantic one: a signature C is called finite if [C]] is a finite category. 

Definition B.1.7. A signature C is called acyclic if it has no non-trivial loops. In other words, for all objects 
c € Ob(C) every path p: c — >■ c is a trivial path. 

Lemma B.1.8 (Acyclicity implies finiteness). If C is an acyclic signature then it is finite. 

Proof. All of our signatures are assumed to include only finitely many objects and arrows. If C is acyclic then 
no edge can appear twice in the same path. Suppose that C has n arrows. Then the number of paths in C is 
bounded by the number of lists in n letters of length at most n. This is finite. 

n 

Lemma B.1.9 (Acyclicity and finiteness are preserved under formation of fiber product). Given a diagram 

F G 

A — > C < — B of signatures such that each is acyclic (respectively, finite), the fiber product signature A Xq B is 
acyclic (respectively, finite). 

Proof. The fiber product of finite sets is clearly finite, and the fiber product of categories is computed by twice 
taking the fiber products of sets (see Definition B.1.4). We now show that acyclicity is preserved. 
Let Coop denote the signature 



Coop : 



(13) 



A category C is acyclic if and only if every functor L : Coop -> C factors through the unique functor Coop — > it, 
i.e. if and only if every loop is a trivial path. The result follows by the universal property for fiber products. 

□ 

Definition B.1.10 (Comma category). To a diagram of categories A ^ C <^ B we can associate the comma 
category of f over g, denoted (fig), defined as follows. 

Ob(/ i g) := {{a, 6, 7) | a G Oh{A),h e Ob(B), 7 : /(a) ^ gih) in C} 

Hom(/;g)((a, 6,7), (a', 6', 7')) := {{m,n) | to : a a' in A, n : 6 ^ 6' in B, 7' o /(to) = g{n) o 7} 

There are obvious projection functors {f \, g) ^ A and (/ ], g) B and a natural transformation a: f op ^ 
g o q, and we summarize this in a canonical natural transformation diagram 

{fig)^^B (14) 

P ), g 
Y 

A 
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Below is a heuristic picture that may be helpful. The outside square consists of categories and functors (note 
that it does not commute!). The inside square represents a morphism in (/ 4- g) from object (0,6,7) to object 
(a', 6', 7'); this diagram is required to commute in C. 

(fig) 



a f{a) - 
a' f{a') 



-gib) b B 

■g{b') b' 



Example B.1.11 (Slice category and coslice category). Let /: A — > _B be a functor, and let b G Oh{B) be an 
object, which we represent as a functor h: — > i?. The slice category of f over b is defined as {f \. b). The 
coslice category of b over F is defined as (6 | /). 

For example let {*} denote a one element set, and let idset be the identity functor on Set. Then the coslice 
category ({*} ], idset) is the category of pointed sets, which we denote by Set*. The slice category (idset i {*}) 
is isomorphic to Set. 

Lemma B.1.12 (Comma categories as fiber products). Given a diagram A ^ C B , the comma category 
if i g) can be obtained as the fiber product in the diagram 



ifig) — 



c- 



ip-i) 



Ax B 



(dom^cod) 



CxC 



where is the category of functors ~t ~^ C. 
Proof. Straightforward. 



Lemma B.1.13 (Acyclicity and finiteness are preserved under formation of comma category). Let A,B, and 



C be acyclic (resp. finite) categories, and suppose we have functors A 
(fig) is acyclic (resp. finite). 



C <^ B. Then the comma category 



Proof. Since the fiber product of finite sets is finite. Lemma B.L12 implies that the formation of comma 
categories preserves finiteness. Suppose that A and B are acyclic (in fact this is enough to imply that (/ I g) 
is acyclic). Given a functor L: Coop — > {fig), this implies that composing with the projections to A and B 
yields trivial loops. In other words L consists of a diagram in C of the form 



-9{b) 



/(«) 



f{a)^^g{b) 

7 



But this implies that 7 = 7', so L is a trivial loop too, competing the proof. 
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B.2 Data migration functors 



Construction B.2.1 (A). Let F: C ^ D he a. functor. We will define a functor Ap: Z?-Inst C-Inst; 
that is, given a D- instance I : D ^ Set we will naturally construct a C-instance Ap{I). Mathematically, this 
is obtained simply by composing the functors to get 



Af{I) := /oF: C-> Set 




To understand this on the level of binary tables, we take F and / as given and proceed to define the C-instance 
J Apl as follows. Given an object c G Ob(C), it is sent to an object d := F{c) G Ob(-D), and I{d) is an 
entity table. Assign J(c) — I{d). Given an arrow g: c — > c' in C, it is sent to a path p — F{g) in D. We 
can compose this to a binary table [p] as described in (5). Assign J{g) = [p]. It is easy to check that these 
assignments preserve the axioms. 

Lemma B.2. 2 (Finiteness is preserved under A). Suppose that C and D are finite categories and F: C ^ D 
is a functor. If I : I? — > Set is a finite D-instance then Apl is a finite C-instance. 

Proof. This follows by construction: every object and arrow in C is assigned some finite join of the tables 
associated to objects and arrows in D, and finite joins of finite tables are finite. 

□ 

Recall the definition of discrete op-fibrations from Definition 4. 

Lemma B.2. 3 (Discrete op-fibrations). Let F : C D be a signature morphism. The following are conditions 
on F are equivalent: 

1. The functor [F]: [C] — >■ [D] is a discrete op-fibration. 

2. For every choice of object cq in C and path q: do ^ dn in D with F(cq) = do, there exists a path p d C 
such that F{p) ^ q, and p is unique up to path equivalence. 

3. For every choice of object cq in C and edge q: do ^ di in D with F{co) — do, there exists a path p G C 
such that F{p) ^ q, and p is unique up to path equivalence. 

Proof. It is clear that condition (2) implies condition (3), and the converse is true because concatenations of 
equivalent paths are equivalent. It suffices to show that (1) and (2) are equivalent. 

It is shown in [IX] that a functor tt: X — > 5 between categories X and 5 is a discrete op-fibration if and 
only if for each solid-arrow square of the form 




where io'.~^ sends the unique object of it to the initial object of T', there exists a unique functor i such 

that both triangles commute. We now translate this statement to the language of signatures with [[C] — X 
and \DJ — S. A functor 1 — > [CJ corresponds to a path in C, and the uniqueness of lift p corresponds to 
uniqueness of equivalence class. This completes the proof. 
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Example B.2.4 (A discrete op-fibration). Let C,D be signatures, and let F: C — > £> be as suggested by the 
picture below. 



61 



91 



ai 



hi 



ci 
■ • 



C 




D 



B 
• - 



A 
• - 



H 



C 
■ • 



This is a discrete op-fibration. 



Corollary B.2.5 (Pre-image of an object under a discrete op-fibration is discrete). Let F : C ^ D he a discrete 
op-fibration and let d e Ob(_D) be an object. Then the pre-image F~^{d) is a discrete signature (i.e. every path 
in F~^(d) is equivalent to a trivial path). 

Proof. For any object d G Oh{D) the pre-image F~^{d) is either empty or not. If it is empty, then it is discrete. 
If F~^{d) is non-empty, let e: cq — >■ ci be an edge in it. Then F{e) = d ^ F(co) so we have an equivalence 
e ^ Co by Lemma B.2.3. 

□ 

Example B.2.6. Let F: A ^ B he a, functor. For any object b £ B, considered as a functor b:lf B, the 
induced functor {b I F) ^ B is a discrete op-fibration. Indeed, given an object b F{a) in {b I F) and a 

morphism h: a — > a' in A, there is a unique map b F(a) — ^—4 F{a') over it. 

Definition B.2.7. A signature C is said to have non-redundant edges if it satisfies the following condition for 
every edge e and path p in C: If e p, then p has length 1 and e = p. 

Proposition B.2.8. Every acyclic signature is equivalent to a signature with non-redundant edges. 

Proof. The important observation is that if C is acyclic then for any edge e and path p in C, if e is an edge in p 
and e ~ p then e = p. Thus, we know that if C is acyclic and e ^ p then e is not in p. So enumerate the edges 
of C as Ec ■— {ei, . . . , e„}. If ei is equivalent to a path in Ec — {ei} then there is an equivalence of signatures 
C ^ C ~ {ei} =: Ci; if not, let Ci := C. Proceed by induction to remove each that is equivalent to a path 
in Ci-i, and at the end no edge will be redundant. 



Corollary B.2.9 (Discrete op-fibrations and preservation of path length). IfC and D have non-redundant edges 

and F: C ^ D is a discrete op-fibration, then for every choice of object cq in C and path q — dg.e^.Cj 

of length n in D with F(ca) — do, there exists a unique path p — cq. 61.62 e„ of length n in C such that 

F(6i) — 6^ for each 1 < i < n, so in particular F{p) — q. 

Proof. Let cq, do, and q be as in the hypothesis. We proceed by induction on n, the length of q. In the base case 
n = then Corollary B.2.5 implies that every edge in F~^{do) is equivalent to the trivial path cq, so the result 
follows by the non-redundancy of edges in C. Suppose the result holds for some n € N. To prove the result for 
rt + 1 it suffices to consider the final edge of q, i.e. we assume that q = e' is simply an edge. By Lemma B.2.3 
there exists a path p G C such that F{p) ~ e', so by non-redundancy in D we know that F{p) has length 1. 
This implies that for precisely one edge in p we have Fiei) = e', and for all other edges in p we have F{ej) 
is a trivial path. But by the base case this implies that p has length 1, completing the proof. 
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Construction B.2.10 (E for discrete opfibrations) . Suppose that F: C — ?> D is a discrete op-fibration. We 
can succinctly define Y^f- C-Inst — > ZJ-Inst to be the left adjoint to Ap, however the formula has a simple 
description which we give now. Suppose we are given F and an instance I: C ^ Set, considered as a collection 
of binary tables, one for each object and each edge in C. We are tasked with finding a _D-instance, J := 
Y.fI: D Set. 

We first define J on an arbitrary object d e Ob(£'). By Corollary B.2.5, the pre-image F~^{d) is discrete in 
C; that is, it is equivalent to a finite collection of object tables. We define J{d) to be the disjoint union 

Jid):= II /(c). 

Similarly, let e: d — >■ cZ' be an arbitrary arrow in D. By Lemma B.2.3 we know that for each c G F^^{d) 
there is a unique equivalence class of paths Pc in C such that F(pc) ~ e. Choose one, and compose it to a single 
binary table — all other choices will result in the same result. Then define J(e) to be the disjoint union 

J(e):= [] I{p,). 

Lemma B.2.11 (Finiteness is preserved under S). Suppose that C and D are finite categories and F: C ^ D 
is a functor. If I : C — >■ Set is a finite C -instance then Y,pl is a finite D instance. 

Proof. For each object or arrow d in D, the table Y,pl(d) is a finite disjoint union of composition-joins of tables 
in C. The finite join of finite tables is finite, and the finite union of finite tables is finite. 

□ 

Remark B.2.12 (E exists more generally and performs quotients and skolemization) . For any functor F: C ^ D 
the functor : Z?-Inst — > C-Inst has a left adjoint, which we can denote by Sj? : C-Inst — ?> _D-Inst because 
it agrees with the constructed in B.2.10 in the case that _F is a discrete op-fibration. Certain queries are 
possible if we can use Si? in this more general case — namely, quotienting by equivalence relations and the 
introduction of labeled nulls (a.k.a. Skolem variables). We do not consider it much in this paper for a few 
reasons. First, quotients and skolem variables are not part of the relational algebra. Second, the set of queries 
that include such quotients and skolem variables are not obviously closed under composition (see Section ??). 

Construction B.2.13 (Limit as a kind of "join all"). Let _B be a signature and let iJ be a S-instance. The 
functor [H] : [B] Set has a limit lim^ H e Ob(Set), which can be computed as follows. Forgetting the path 
equivalence relations, axioms (3) and (4) imply that H consists of a set {A^i, . . . ,iVm} of node tables, a set 
{ei, . . . , e„} of edge tables, and functions s, t : {1, . . . , n} — >■ {1, . . . , m} such that for each \ < i < n the table 
constitutes a function : Ng^i) — > Nt(_i). For each i define Xi as follows: 

Xi := 7ri^2, 

■= T^l,2,...m<^s{i)=m+l'^t{i)=m+2{^i-l ^ Ci) 
Xn '■— T^l,2,...mO's{n)=m+lO't(n)=m+2{Xn-l X e„) 

Then X := Xn is a table with m columns, and its set \X\ of rows (which can be constructed if one wishes by 
concatenating the fields in each row) is the limit, |^| = lim^ H . 

Construction B.2.14 (11). Let F: C — D be a signature morphism. We can succinctly define Hp : C-Inst 
D-Inst to be the right adjoint to A^?, however the formula has an algorithmic description which we give now. 
Suppose we are given F and an instance / : C — > Set, considered as a collection of binary tables, one for each 
object and each edge in C. We are tasked with finding a Z)-instance, J := Hpl: D — > Set. 

We first define J on an arbitrary object d: — > 13 (see Example B.1.3). Consider the comma category 
Bd {d I F) and its projection qd'. {d I F) — > C. Note that in case C and D are finite, so is Bd. Let 
Hd := I o Qd: Bd ^ Set. We define J{d) :— lim^^ Hd as in Construction B.2.13. 



...,2m{Ni X • • • X N„i) 
...mCrs{l)=m+l'^t{l)=m+2iXo X ^l) 
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Now let e : d — > d' be an arbitrary arrow in D. For ease of notation, rewrite 



B:=B,i, B':=Bd', q := qa, q':=qd', H := H^, andH':=Hd'. 



We have the following diagram of categories 



id' i F) 



(dlF) 




Set 



A unique natural map J{d) = linis H — > linis H' = J{d') is determined by the universal property for limits, 
but we give an idea of its construction. There is a map from the set of nodes in B' to the set of nodes in B and 
for each node N in B' the corresponding node tables in H and H' agree. Let Xi and X'^ be defined respectively 
for {B,H) and {B',H') as in Construction B.2.13. It follows that Xq is a projection (or column duplication) 
on Xq. The set of edges in B' also map to the set of edges in B and for each edge e in B' the corresponding 
edge tables in H and H' agree. Thus the select statements done to obtain J(d') — X' contains the set of select 
statements performed to obtain J{d') = X. Therefore, the map from J{d) to J{d') is given as the inclusion of 
a subset followed by a projection. 

If C or Z) is not finite, then the right pushforward Hp of a finite instance / G C-Set may have infinite, even 
uncountable results. 

Example B.2.15. Consider the unique signature morphism 



Here [2?] has arrows {/" | n e N} so it is infinite. In this case {d I F) is the discrete category with a countably 
infinite set of objects Ob(d J, -P") - N. 

Given the two-element instance / : C — Set with I{s) = {Alice, Bob}, the rowset in the right pushforward 
IIf{I) is the (uncountable) set of infinite streams in {Alice, Bob}, i.e. 



Proposition B.2.16. Let H: B ^ Set be a functor, and let q: B ^ he the terminal functor. Noting that 
there is an isomorphism of categories -Set = Set, we have a bijection limg H = HqH. 

Proof. Obvious by construction of 11. 



Proposition B.2.17 (Behavior of A,E,n under natural transformations). Let C and D be categories, let 
F,G: C D be functors, and let a: F G be a natural transformation as depicted in the following diagram: 




IIf{I){s) = lis) 



□ 



F 



c 




D 



G 



Then a induces natural transformations 



A„: Af ^ Ag, 



Sc : Sg — t' Yip, 



and 
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Proof. For any instance J: D ^ Set and any object c G Ob(C) we have ac'- J o F{c) — > J o G(c), and the 
naturality of a imphes that we can gather these into a natural transformation Aa{J) : Af{J) — > Ag{J)- One 
checks easily that this assignment is natural in J, so we have Aq : Ap ~^ Aq as desired. 

Now suppose that I : C Set is an instance on C . Then for any J e DSet we have natural maps 

Hom(J,nG/) = Hom(AGJ,/) ^ Hom(Ai.J,/) s Hom(J,nF/) 
so by the Yoneda lemma, we have a natural map Hq — > 11^, as desired. We also have natural maps 

Hom(Ej./, J) s Hom(/, ApJ) ^ Hom(/, Ac J) = Hom(EG-^, J) 
so by the Yoneda lemma, we have a natural map Eg — >■ "^f, as desired. 



B.3 Query composition 

In this section and those that follow we were greatly inspired by, and follow closely, the work of [9] . While their 
work does not apply directly, the adaptation to our context is fairly straightforward. 

Lemma B.3.1 (Unit and counit for S are Cartesian). Let F: C ^ D be a discrete op-fibration, and let 

rj: idc-inst ^ Ap^F and e: Y.fAf idu-inst 

be (respectively) the unit and counit of the (T,f,Af) adjunction. Each is a Cartesian natural transformation. 
In other words, for any morphism a: I ^ I' of C -instances and for any morphism h: J ^ J' of D -instances, 
the induced naturality square (left below for the unit and right below for the counit) 



I 



r 



EfAf(J) ^"^"^^^ Y.FAFiJ') 



Vi' 



Af^f{I) 



■AF^Fin 



is a pullback in C-Inst and D-Inst respectively. 



J' 



Proof. It suffices to check that for an arbitrary object c €E Ob(C) and d G Oh{D) respectively, the induced 
commutative diagram in Set 



m 

iVl)o 

AF^FiI){c) 



■I'{c) 
■Af^f{I'){c) 



EFAF{J){d) ^^^^^^FAF{J'){d) 



AFSF(a)c 

is a pullback. Unpacking definitions, these are; 



(ej)d 
J(d) 



-J'{d) 



/(c) 



/'(c) 



{c' I F(c')=F{c)} " {d I F{c')=F{c)} 



II J{d) 

{c I F(c)=d} 



J{d) 



II J'{d) 

{c I F(c)=d} 



J'(d) 



Roughly, the fact that these are puUbacks squares follows from the way coproducts work (e.g. the disjointness 
property) in Set, or more precisely it follows from the fact that Set is a positive coherent category [12, p. 34]. 
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Definition B.3.2 (Grothendieck construction). Let C be a signature and I: C ^ Set an instance. The 
Grothendieck category of elements for I over C consists of a pair (J^/, tt/), where I is a signature and 
TT/ : / — > C is a signature morphism, constructed as follows. The set of nodes in I is {(c, x) | c G 
Oh{C),x e I{c)}. The set of edges in I from node {c,x) to node {c\x') is {e: c — >■ c' | I{e){x) = x'}. The 
signature morphism tt/ : / ^ C is obvious: send (c, x) to c and send e to e. Two paths are equivalent in I 
if and only if their images under tt/ are equivalent. We sometimes denote simply as J. 

Lemma B.3.3. Let I: C ^ Set be an instance. Then ttj: J I ^ C is a discrete op-fibration. 

Proof. This follows by Lemma B.2.3. 

□ 

Lemma B.3.4 (Acyclicity and finiteness are preserved under the Grothendieck construction). Suppose that C 
is an acyclic (respectively, a finite) signature and that I: C ^ Set is a finite instance. Then J I is an acyclic 
(respectively, a finite) signature. 

Proof. Both are obvious by construction. 

□ 

Definition B.3.5 (DeGrothendieckification) . Let tt: X C he a. discrete op-fibration. Let {*}^ denote a 
terminal object in X-Inst, i.e. any instance in which every node table and edge table consists of precisely one 
row. Define the deGrothendieckification of tt, denoted cJtt: C — > Set to be dn := (^{*}"^^ S C-Inst. 

One checks that for a discrete op-fibration tt : X — > C and object c e Ob(C) we have the formula 

c?7r(c) = 7r^"^(c), 

so we can say that deGrothendieckification is given by pre-image. 

Lemma B.3.6 (Finiteness is preserved under DeGrothendiekification). If X is a finite signature and tt: X ^ C 
is any discrete op-fibration, then Ott is a finite C-instance. 

Proof. Obvious. 

□ 

Proposition B.3.7. Given a signature C, let Dopfc C Cat/p denote the full category spanned by the discrete 
op-fibrations over C. Then J: C-Inst — >■ Dopfc o-i^d d: Dopfc ~^ C-Inst are functorial, d is left adjoint to 
J , and they are mutually inverse equivalences of categories. 

Proof. See [18, Lemma 2.3.4, Proposition 3.2.5]. 

□ 

Corollary B.3.8. Suppose that C and D are categories, that F,G: C — > D are discrete op-fibrations, and 
a: F G is a natural transformation. Then there exists a functor p: C ^ C such that F o p = G, and 
Eq = dip), where Sq, : So — > Sj? is the natural transformation given in Proposition B.2.17. 

Proof. By Proposition B.2.17 the natural transformation a: F ^ G induces a natural transformation — ^ S^?, 
and applying deGrothendieckification supphes a map dG — > dF. By Proposition B.3.7 this induces a map 
p: C ^ G over D with the above properties. 

□ 

Proposition B.3.9. Given a commutative diagram 




in which g and h are discrete op-fibrations, it follows that f is also a discrete op-fibration. 
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Proof. Consider the diagram to the left below 





— > fa 

> B 




■C 



Then since h is a, discrete op-fibration there exists a unique i: 1 — > A making the middle diagram commute. 
But now we have two lifts, q and fi, and since 5 is a discrete op-fibration, it follows that q = f£, completing 
the proof. 



Proposition B.3.10 (Discrete op-fibrations are stable under puUback). Let F: C ^ C be a functor and 
t: : X ^ C be a discrete op-fibration. Then given the pullback square 



C" xc'X- 

J 

c 

F 

the map tt' is a discrete op-fibration, and there is a natural isomorphism 

Ap{dn). 



Proof. A functor p: Y ^ D is a discrete op-fibration if and only there exists a functor d: D ^ Set such that 
the diagram 




Set 



is a pullback square, where Set* is the category of pointed sets as in Example B.1.11. The result follows from 
the pasting lemma for fiber products, and Lemma B.3.3. 

□ 

Proposition B.3.11. The complexity of checking that a functor is a discrete op-fibration is a combination of 
the complexity of checking equivalence of categories and the complexity of the word problem for categories (which 
is r.e.). 

Proof. Any equivalence of categories is a discrete op-fibration, so one containment is trivial. Suppose that 
F : C — > Z? is a signature morphism. Let C be the signature with the same objects and arrows as C and with 
all the path equivalences in C, plus a path equivalence p ^ p' whenever F{p) ~ F(p') in D. Then F factors as 
a composition 




and F is a discrete op-fibration if and only Hi: C ^ C is an equivalence of categories. 
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Proposition B.3.12 (Comma Beck-Chevalley for S,A). Let F: C ^ D and G: E ^ D be functors, and 
consider the canonical natural transformation diagram (I4) 



(F i G) 



E 



C 



Y 

D 



Let he the generalized left push-forward as defined in Remark B.2.12. Then there is an isomorphism 

T.gAp A Ag^f 

of functors C-Inst — > i?-Inst. 

Proof. The map is given by the composition 



Ag^f 



To prove it is an isomorphism, one checks it on each object e G Oh{E) by proving that the diagonal map 
(F I Ge) — > (g I e) in the diagram 




is a final functor, and this is easily checked. 



Corollary B.3.13 (Comma Beck-Chevalley for A, 11). Let F : C ^ D and G: E ^ D be functors, and consider 
the canonical natural transformation diagram (I4) 



{F i G) 



E 



G 



C 



Y 

D 



Then there is an isomorphism 



AfUg ^ HpA, 



of functors F-Inst -^> C-Inst. 

Proof. This follows from Proposition B.3.12 by adjointness. 



Sometimes when we push forward an instance / along a functor B — > C, the resulting instance Sg/ agrees 
with /, at least on some subcategory A C B. This is very useful to know, because it gives an easy way to 
calculate row sets and name rows in the pushforward. The following lemma roughly says that this occurs if for 
all b £ Ob(i?) and a £ Oh{A), the sense to which b is to the left of a in C, is the same as the sense to which b 
is to the left of a in S. 
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Lemma B.3.14. Suppose that we are given a diagram of the form 



A 

F 

Then the map (3: — > A^Sg is an isomorphism if, for all objects a € Ob(^) the functor 



G : (idsiqa) ^ {G!^Fa) 



is final. 



Proof. For typographical convenience, let [0] denote the terminal category, usually denoted . To check that 
/3 is an isomorphism, we may choose an arbitrary object a : [0] — > A and check that Aa/3 is an isomorphism. 
Consider the diagram 

(G i Fa) 




The map f3 is the composite 



AaA, s EtApAq = E„Eg'Ag'A„ 



"SuAy = AaA^Ec, 



and this is equivalent to showing that /3' is an isomorphism. It suffices to show that Et — > E,iA„ is an 
isomorphism, but this is equivalent to the assertion that G" is a final functor. One can factor G" as 

(id^ I a) — > (ids | qa) — > (G I Fa) 

and the first of these is easily checked to be final. Thus the composite is final if and only if the latter map is 
final, as desired. 

□ 

Proposition B.3.15 (Fullback Beck-Chevalley for E, A when E is along a discrete op-fibration) . Suppose that 
F: D ^ C is a functor and p: X ^ C is a discrete op-fibration, and form the pullback square 



Y 



■X 



D 



G 



Then there is a natural isomorphism 

of functors X-Inst D-Inst. 

Proof. The map is given by the composition 



E^Ag ^ ApE 



E„A 



■ EgAGApEp 



■ EqAgAi^Ep — —s^ Ap'Sp 



Since rjp and are Cartesian (by Lemma B.3.1), it suffices to check this on the terminal object of X-Inst, 
where it follows by Proposition B.3.10. 
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Corollary B.3.16 (Fullback Bcck-Chevalley for A, 11 when A is along a discrete op-fibration). Suppose that 
F: D C is a functor and p: X ~> C is a discrete op-fibration, and form the pullhack square 



Y 



X 



D 



C 



Then there is a natural isomorphism 

HgA^ a ApHF 

of functors I? -Inst — > X Inst. 

Proof. This follows from Proposition B.3.15 by adjointness. 



Proposition B.3.17 (Distributive law). Let u: C ^ B be a discrete op-fibration and f:B^A any functor. 
Construct the distributivity diagram 

(15) 




as follows. Form v: M ^ A by v := J^Ilf{du), form w: N ^ B by w :— Jg AfUfdu. Finally let e: N ^ C 
be given by Jg ef{du), where ef. A/II/ — >■ ids is the counit. Note that N = B x a M by Proposition B.3.10 and 
that one has w — uo e by construction. 
Then there is a natural isomorphism 



(16) 



of functors C-Inst — > A-Inst. 



Proof. We have that u^v^ and w are discrete op-fibrations. By Corollary B.3.16 we have the following chain of 
natural transformations 



n/E„. 



Every natural transformation in the chain is Cartesian, so it suffices to check that this map is an isomor- 
phism when applied to the terminal object l*}'^ in C~Inst. But there the composition is simply the identity 
transformation on Yifidu), proving the result. 



Example B.3.18 (Why S-restrictedness appears to be necessary for composability of queries). Without the 
condition that it be a discrete op-fibration, Proposition B.3.17 does not hold. Indeed, let C — Coop as in (13) 
and let _B = yl = be the terminal category. One finds that N — M ~ too in (15), and e is the unique 
functor. If one considers C-sets as special kinds of graphs (discrete dynamical systems) then we can say that 
UfTiu will extract the set of connected components for a C-set, whereas E^IIgAe will extract its set of vertices. 

We may still wish to ask "whether a 11 can be pushed past a S", i.e. whether for any C —> B A, 
some appropriate v,g,e exist such that an isomorphism as in (16) holds. The following example may help give 

intuition. Let C —> B —> A he given as follows 























J) 


u 




/ 












V 


• _^ • 








m 
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where u{e) = v and /(r) = v. The goal is to find some C ^ iV A M A- A such that isomorphism (16) holds. 
This does not appear possible if M and N are assumed finitely presentable. 



Theorem B.3.19 (Query composition). Suppose that one has Yi-restricted data migration queries Q. S ^ T 
and Q' : T U as follows: 

B^-^A D—^C (17) 




S Q T T Q' U 



Then there exists a Y,-restricted data migration query Q" : S U such that [Q"]] = IQ']] o [[Q]. 

Proof. We follow the proof in [9], as we have been throughout this section. We will form Q" by constructing 
the following diagram, which we will explain step by step: 

N ^ ^D'^-^M (18) 




S T U 



First, form (i) by taking the puUback; note that fc is a discrete op-fibration by Proposition B.3.10. Second, form 

(ii) as a distributivity diagram (see Proposition B.3.17) and note that w is a discrete op-fibration. Third, form 

(iii) and (iv) with B' = {f i h) and TV = (r i e). 

By Proposition B.3.15 we have A„Et = SfcA^. By Proposition B.3.17 we have IlgEfe = E^jIIqAe. By 
Corollary B.3.13 we have both A/jII/ = lir^m and AJlr - npA„. Pulling this all together, we have an 
isomorphism 

s,ngA„Stn/A, - s^ngSfeA^n/A, - E,s^,n,AeA,,n/A, 

— J^tj^^ujIIg Aell^ Am A^ 
= S^S^ngllpAj^ A^ As , 

which proves that \Q"J = IQ'l o [QJ, where Q" is the data migration functor given by the triple of morphisms 
Q' := {s o m o n,qo p^v o w), i.e. [QJ = Yt^w^qp^smn- This completes the proof. 

□ 

Corollary B.3.20. Suppose that Q: S ^ T and Q' : T U are queries as in (17) and that both are (A,S)- 
restricted. Then there exists a (A,Y,)-restricted data migration query Q" : S U such that [Q"] = IQ'J o |[Q]|. 

Proof. We form a diagram similar to (18), as follows. First, form (i) by taking the puUback; note that k is 
a discrete op-fibration by Proposition B.3.10. Second, form (ii) as a distributivity diagram (see Proposition 
B.3.17) and note that w is a discrete op-fibration. Third, form (iii) and (iv) as puUbacks. Note that h, m, and 
n will be discrete opfibrations. Following the proof of Theorem B.3.19, the result follows by Corollary B.3.16 in 
place of Corollary B.3.13. 

□ 

B.4 Syntactic characterization of query morphism and query equivalence 

We continue to closely follow [9]. 
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Definition B.4.1 (Morphism of data migration queries). A morphism of {A,'E,)-restricted data migration 
queries from P = {u, g, v) to Q = (s, /, t), denoted (c, a, /3) : P ^ Q, consists of two functors c, a and a natural 
transformation /3, fitting into a diagram of the form 



P 



Q 



s- 



s- 



D 

A 



BxaC 



Y 

B 



f 



C 



C 



A- 



■T 



(19) 



T 



where blue arrows {u,v^ s,t, c) are required to be discrete op-fibrations and every square commutes except for 
the top middle square in which /3: /' —> 5a is a natural transformation. 

Note that for any diagram of the form (19), the maps a and b will automatically be discrete op-fibrations 
too. 

Remark B.4.2. Note that a morphism of queries P ^ Q involves a "backward" morphisms of schemas, a: B Xa 
C — > £> as in (19). In fact, this reversal happens in the conjunctive fragment only (see Lemma B.4.9). This 
may not be surprising because a is a morphism of indexing categories, or "bound variables", and thus the 
directionality is in line with classical database theory. 

Definition B.4.3 (Composition of data migration queries). Suppose given data migration queries P,Q,R : S 
T and morphisms P ^ Q and Q — > i? as in Definition 19. Then they can be composed to give a query morphism 
P — 7> R, as follows. 

We begin the following diagram 



S- 



F ■ 



DxcE- 



D 



B xaC- 

b 

— B 



/' 



E- 



■E 



C 



C 



A- 



■T 



■T 



■T 



By Proposition B.2.17 there is a canonical map /' x^e — > {go) x^e, and this allows us to construct a composite 



F 



BxaE 



B 



E 



E 



A 



f 

Remark B.4.4. Note that this composition is associative up to isomorphism, but not on the nose. Thus we 
cannot speak of the category of data migration queries P Q, but only the (00, l)-category of such. 
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The following definition is slightly abbreviated, i.e. not fully spelled out, but hopefully it is clear to anyone 
who is following so far. 

Definition B.4.5 (Category of data migration queries). We define the category of (A,Y.)-restricted data mi- 
gration queries from S to T, denoted RQry(S', T), to be the category whose objects are (A, S)-restricted data 
migration queries S T and whose morphisms are equivalence classes of diagrams as in (19), where two such 
diagrams are equivalent if there are equivalences of categories between corresponding objects in their middle 
rows and the functors commute appropriately. 

Lemma B.4.6. Given a morphism of (A,Y.)-restricted data migration queries P ^ Q, there is an induced 
natural transformation [P] — >■ HQ]]. 

Proof. By Proposition B.2.17 and Corollary B.3.16 we have the natural natural transformations below: 



E,ngA„ = EtE,ngA„ ^ StScHgn^AaA, (20) 

A EtS,n/-A,A„ 
= StEeH/^AbA, 
= StScA,n/A, ^StH/A, 

□ 

Lemma B.4.7. Let Q: S T be a {A ^Y,) -restricted data migration query, and let P: S'-Inst — >■ T-Inst be 
any functor. If there exists a Cartesian natural transformation (p: P ^ IQl, then there is an essentially unique 
{A,Tj)-restricted data migration query isomorphic to P. 

Proof. Suppose that Q is represented by the bottom row in the diagram below 

Q' : S-^^BxaC^^^C^--^T (21) 

J 

b c 

Y y 
Q: B 

We form the rest of the diagram as follows. Let * be the terminal object in S'-Inst, so in particular JrpQ{*) = t. 
Let t' = Jj, P{*) and define c: t' ^ t over T to be Jj, <j>{*). Let b be the puUback of c and s' — sob. The Diagram 
(21) now formed, let Q' : 5 T be the top row. Note that c, and therefore b and s, are discrete op-fibrations, 
so in particular Q' is a (A, E)-restricted data migration query. Note also that there was essentially no choice in 
the definition of Q'. 

The map |[Q'l IQJ is given by 

St-n/-A,, = EtE,n/,AbA, A StSeA,n/A, A StH/A,, 

so it is Cartesian by Lemma B.3.1. But if both P and Q' are Cartesian over Q and if they agree on the terminal 
object (as they do by construction), then they are isomorphic. Thus P = Q' is a (A, I])-restricted data migration 
query. 

n 

Lemma B.4.8 (Yoneda Functorialization). Let C be a category and let b: C ^ Set be a functor with s: B C 
its Grothendieck category of elements. Then there is a natural isomorphism of functors 

Homc-set(^-) = ntnsA« 

where t: I is the unique functor. Moreover, for any b' : C ^ Set with s' : B' ^ C its category of elements, 

there is a bisection 

Homcat/c(S,P') =Homc-Set(6,fo') =Hom(ntn,,A,,,ntn,A,). 
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Proof. The second claim follows from the first by Proposition B.3.7 and the usual Yoneda-style argument. For 
the first, we have 

n„n,A,(-) -Homset(*,n„n,A,(-)) -HomB_set(*'',As(-)) =Homc.-set(S3(*^),-) 

= Home-Set (^,-)- 



Lemma B.4.9. Suppose given the diagram to the left, where u and u' are discrete op-fibrations: 



S- 



D 



■C 



s- 



X 



c 



A 

a I3i^ 
-X 



■ c 



G 



and a natural transformation j: IlgA^ — > HgiA^i. Then there exists an essentially unique diagram as to the 
right, such that j is the composition (see Proposition 3.2.11), 



(22) 



Proof. Choose an object c: — > C in C. Form the diagram 







S 



C- 






X 



Let s = u o p and s' = u' o p', and let t : — > denote the unique functor. It is easy to show that p and p' , 
and hence s and s' are discrete op-fibrations. We have a natural transformation 



By Lemma B.4.8 this is equivalent to giving a map jc'. (c g') — >■ (c g) over 5*. Moreover, given a map 
/: Co — >■ ci in C, the natural transformation A/: Ac^ — >■ A^ (see Proposition B.2.17) induces a commutative 
diagram of functors S Set, as to the left 



Ae,ng/A„. Ac.HgA^ 



A,„ng,A„. -.^ A^^UgAu 



(ci i g') (ci i g) 

f f 
(co i g') — ^ (co i g) 



which induces a commutative diagram of categories over S as to the right. 

We are now in a position to construct a unique functor a: X ^ D with u' = uoa and natural transformation 
/3: g' ^ g o a agreeing with each j^.. Given an object x £ Oh{X), we apply the map jgi^ : {g'x \. g') — > {g'x \. g) 

to {x,idgix) to get some {d,g'x \ gd). We assign a{x) :— d and f3x '.— b, and note that u'(x) = u{d) as above. 
A similar argument applies to morphisms, so a and /3 are constructed. To see that j is the composite given in 
(22), one checks it on objects and arrows in C using the formula from Construction B.2.14 in conjunction with 
Proposition B.2.16. 
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Theorem B.4.10. Suppose that P,Q '. S T are (A, Y,) -restricted data migration queries, and X : [P] — >■ [[Q] 
is a natural transformation of functors. Then there exists an essentially unique morphism of {A, Yi) -restricted 
data migration queries x: P ^ Q (i.e. a diagram of the form (19)) such that [xj = X. In other words, the 
functor RQry(S', T) — >■ Fun(S', T) is fully faithful. 

Proof. Let * denote the terminal object in 5-Inst. We define a functor P: S-Inst ^> T-Inst as follows. For 
I: S Set, define P{I) as the fiber product 

P{i) IQW) 

Y T 

There is a induced natural transformation i: ^PJ — > P and an induced Cartesian natural transformation 
X':P^IQJ, such that X = X' oi. _ 

Let v: C ^ T and t: A ^ T denote the category of elements v := JrplPl{*) = Jt Pi*) t := /^IQK*) 
respectively, and let c :— Jj, X{*) : w — > i over T. If P and Q are the top and bottom rows in the diagram below, 
then Lemma B.4.7 and in particular Diagram (21) implies that there is an essentially unique way to fill out the 
middle row, and its maps down to the bottom row, as follows: 

P: S^-^ D ^ — ^C—^T 

P' : S B XaC ^^-^C —^T 

J 

6 c 

Q: B 

with s' = s o 5 and [Pi = P. 

Lemma B.3.1 implies that i: E„ngA„ — > HyHfAs' induces a natural transformation j: UgA-u — >■ UfAs' 
with Yyj — i. The result follows by Lemma B.4.9. 

□ 

Corollary B.4.11. Let f,g: S ^ T be { A, Y) -restricted data migration queries. Then the complexity of 
I/l — Isl is at most the complexity of testing equivalence of categories. 

Proof. By Theorem B.4.10 proving that [/J = [gj is equivalent to proving that f = g, and by Definition B.4.5, 
we need only check that certain functors are equivalences of categories. 

□ 
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